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ABSTRACT 

Human hands are delicate instruments. Hand gestures 
and finger gestures are excellent ways of emphasizing 
what we say, but on the other hand they can also 
reveal our true intentions. In this paper introduced a 
continuous Indian sign language recognition system, 
wherever each the hands are used for playacting any 
gesture. Recognizing a sign language gestures from 
continuous gestures could be a terribly difficult 
analysis issue. In this paper, a new skeleton-based 
approach is proposed for 3D hand gesture recognition. 
Specifically, we exploit the geometric shape of the 
hand to extract an effective descriptor from hand 
skeleton connected joints returned by the Intel 
RealSense depth camera. This paper solve the 
problem using gradient based key frame extraction 
technique. These key frames are useful for splitting 
continuous language gestures into sequence of signs 
further as for removing uninformative frames. After 
splitting of gestures every sign has been treated as 
associate degree isolated gesture. Then features of 
pre-processed gestures are extracted using orientation 
histogram (OH) with principal component analysis 
(PCA) is applied for reducing dimension of features 
obtained after OH. 

Keywords: Gesture Recognition, Orientation 

histogram (OH); Correlation; Indian sign language 
(ISL). 

1. INTRODUCTION 

Sign languages (also known as signed languages) 
are languages that use manual communication to 
convey meaning. This can include simultaneously 
employing hand gestures, movement, orientation of 
the fingers, arms or body, and facial expressions to 
convey a speaker's ideas. Sign languages often share 


significant similarities with their respective spoken 
language, such as American Sign Language (ASL) 
with American English. Grammar and sentence 
structure, however, may vary to encourage efficiency 
and fluidity in speaking.[1][2] It is important to note 
that just because a spoken language is intelligible 
transnationally, such as English in the United States 
and the United Kingdom, does not mean that the sign 
languages from those regions are as well; ASL and 
British Sign Language (BSL) were formed 
independently and are therefore unintelligible.[3] 

Humans know each other by conveying their ideas, 
thoughts, and experiences to the people around them. 
There are numerous ways to achieve this and the best 
one among the rest is the gift of Speech. Through 
speech everyone can very convincingly transfer their 
thoughts and understand each other. It will be 
injustice if we ignore those who are deprived of this 
invaluable gift. The only means of communication 
available to the vocally disabled is the use of sign 
language. Using sign language they are limited to 
their own world. This limitation prevents them from 
interacting with the outer world to share their feelings, 
creative ideas and potentials. Very few people who 
are not themselves deaf ever learn to sign language. 
These limitation increases the isolation of deaf and 
dumb people from the common society. Technology 
is one way to remove this hindrance and benefit these 
people. Several researchers have explored these 
possibilities and have successfully an achieved finger 
spelling recognition with high levels of accuracy. But 
progress in the recognition of sign language, as a 
whole has various limitations in today s applications. 
The problem of automated sign language recognition 
can be put across as, given a video of a sign language 
sentence, can We identify the signs in the sentence 
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and reconstruct the sentence? The solution to the 
problem of sign language recognition has many 
practical implications. Firstly, advances in automated 
sign language recognition are necessary to improve 
the quality of life of deaf persons by facilitating their 
interaction with hearing populace in public situations. 

The script of Kannada language is syllabic. The 
language uses forty nine phonemic letters which are 
segregated into three groups- Swaragalu, the vowels, 
the Vyanjanagalu, the consonants and 
Yogavaahakagalu, the two characters which are not 
vowel or consonant. The Character set is very similar 
to that of other Indian languages. The script is fairly 
complex as like other complex scripts it has also been 
derived from Brahmi script. As far as Kannada 
Grammar is concerned, it is a highly inflected 
language with three genders- the masculine, feminine 
and the neutral, there are two numbers-singular and 
plural. Kannada is inflecting for gender, number and 
tense, among other things. A primary objective of this 
work is to recognize sign and generate equivalent text 
description in Kannada language. The significant 
contribution made in this paper helps in translation of 
sign language to text I Kannada language for single 
signer. The present work attempts to recognize the 
sign based on shape and features. The extracted 
features are input to ANN Classifier independently. 
The study leves scope for further research on 
translating different signer of sign language into text 
information in Kannada language with grammatically 
correct meaning. The literature on related woks is 
reviewed to know the state of the art and survey is 
organized. 
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Shows a kannada sign symbol basic letter (16 ) 


2 A Review of Previous Research 

Sign language continues to be the preferred method of 
communication among the deaf and the hearing- 
impaired. Advances in information technology have 
prompted the development of systems that can 
facilitate automatic translation between sign language 
and spoken language. More recently, systems 
translating between Arabic sign and spoken language 
have become popular. This paper reviews systems and 
methods for the automatic recognition of Arabic sign 
language. Additionally, this paper highlights the main 
challenges characterizing Arabic sign language as 
well as potential future research directions [1], In this 
paper a Sign Language Recognition system has been 
proposed. The first step of this system is to create a 
database of Indian Sign Language. This is done by 
acquiring the videos from the signers while they are 
performing the hand gestures. Next step is Hand 
tracking and Segmentation. This is performed in order 
to extract features from a particular gesture. A three 
step algorithm has been used in the proposed system 
to get better quality hand tracking and segmentation. 
This algorithm works on motion tracking, edge 
detection and skin colour detection. The system is 
implemented successfully and results are presented in 
this paper. The results demonstrate working of motion 
tracking, edge detection and skin colour detection 
individually as well as their combined effect [2], 

It is difficult for most of us to imagine, but many who 
are Deaf-mute rely on sign language as their primary 
means of communication. They, in essence, hear and 
talk through their hands. Sign languages are visual 
languages. They are natural languages which are used 
by many deaf mute people all over the world. In sign 
language the hands convey most of the information. 
Hence, vision -based automatic sign language 
recognition systems have to extract relevant hand 
features from real life image sequences to allow 
correct and stable gesture classification. In our 
proposed system, we intend to recognize some very 
basic elements of sign language and to translate them 
to text. Firstly, the video shall be captured frame by 
frame, the captured video will be processed and the 
appropriate image will be extracted, this retrieved 
image will be further processed using BLOB analysis 
and will be sent to the statistical database; here the 
captured image shall compared with the one saved in 
the database and the matched image will be used to 
determine the performed alphabet sign in the 
language. Here, we will be implementing only 
American Sign Language Finger spellings, and we 
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will construct words and sentences with them [3], In 
the recent years many approaches have been made 
that uses computer vision algorithms to interpret sign 
language. This endeavour is yet another approach to 
accomplish interpretation of human hand gestures. 
The first step of this work is background subtraction 
which achieved by the Euclidean distance threshold 
method. Thinning algorithm is then applied to obtain 
a thinned image of the human hand for further 
analysis. The different feature points which include 
terminating points and curved edges are extracted for 
the recognition of the different signs. The input for the 
project is taken from video data of a human hand 
gesturing all the signs of the American Sign Language 

[4]. 

In recent years, enormous research is progressing in 
the field of computer vision and human computer 
interaction where hand gestures play a vital role. 
Hand gestures are more powerful means of 
communication for hearing impaired when they 
communicate to the normal people everywhere in day 
to day life. As the normal people find little difficulty 
in recognizing and interpreting the meaning of sign 
language expressed by the hearing impaired, it is 
inevitable to have an interpreter for translation of sign 
language. To overcome this difficulty, an automatic 
hand gesture recognition system which translates the 
sign language into text needs to be developed. In this 
paper, a static hand gesture recognition system for 
American Sign Language using Edge Oriented 
Histogram (EOH) features and multiclass SVM is 
proposed. The edge histogram count of input sign 
language alphabets is extracted as the features and 
applied to a multiclass SVM for classification. The 
average accuracy of the system is compared with 
different number of features and the experimental 
findings demonstrate that the proposed method gives 
a success rate of 93.75% [5]. Visual Interpretation of 
gestures can be useful in accomplishing natural 
Human Computer Interactions (HCI). In this paper we 
proposed a method for recognizing hand gestures. We 
have designed a system which can identify specific 
hand gestures and use them to convey information. At 
any time, a user can exhibit his/her hand doing a 
specific gesture in front of a web camera linked to a 
computer. Firstly, we captured the hand gesture of a 
user and stored it on disk. Then we read those videos 
captured one by one, converted them to binary images 
and created 3D Euclidian Space of binary values. We 
have used supervised feed forward neural net based 
training and back propagation algorithm for 
classifying hand gestures into ten categories: hand 


pointing up, pointing down, pointing left, pointing 
right and pointing front and number of fingers user 
was showing. We could achieve up to 89% correct 
results on a typical test set [6]. 

This paper presents an automatic translation system of 
gestures of the manual alphabets in the Arabic sign 
language. The proposed Arabic Sign Language 
Alphabets Translator (ArSLAT) system does not rely 
on using any gloves or visual markings to accomplish 
the recognition job. As an alternative, it deals with 
images of bare hands, which allows the user to 
interact with the system in a natural way. The 
proposed system consists of five main phases; 
preprocessing, best-frame detection, category 
detection, feature extraction and classification. The 
extracted features used are translation, scale, and 
rotation invariant, which make the system more 
flexible. Experiments revealed that the system was 
able to recognize the 30 Arabic alphabets with an 
accuracy of 91.3% [7], This paper presents a novel 
technique for hand gesture recognition through human 
computer interaction based on shape analysis. The 
main objective of this effort is to explore the utility of 
a neural network-based approach to the recognition of 
the hand gestures. A unique multi-layer perception of 
neural network is built for classification by using 
back-propagation learning algorithm. The goal of 
static hand gesture recognition is to classify the given 
hand gesture data represented by some features into 
some predefined finite number of gesture classes. The 
proposed system presents a recognition algorithm to 
recognize a set of six specific static hand gestures, 
namely: Open, Close, Cut, Paste, Maximize, and 
Minimize. The hand gesture image is passed through 
three stages, pre-processing, feature extraction, and 
classification. In pre-processing stage some operations 
are applied to extract the hand gesture from its 
background and prepare the hand gesture image for 
the feature extraction stage. In the first method, the 
hand contour is used as a feature which treats scaling 
and translation of problems (in some cases). The 
complex moment algorithm is, however, used to 
describe the hand gesture and treat the rotation 
problem in addition to the scaling and translation. The 
algorithm used in a multi-layer neural network 
classifier which uses back-propagation learning 
algorithm. The results show that the first method has a 
performance of 70.83% recognition, while the second 
method, proposed in this article, has a better 
performance of 86.38% recognition rate [8]. 
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3-PROPOSED SYSTEM 

3.1 Dynamic Hand Gesture dataset (DHG- 14/28 

Skeleton-based action recognition approaches have 
become popular as Shotton et al. [12] proposed a real¬ 
time method to accurately predict the 3-D positions of 
body joints from depth images. Hence, several 
descriptors in the literature proved how the position, 
motion, and orientation of joints could be excellent 
descriptors for human actions. Collected datasets for 
action recognition purpose like [17, 9] provide 
usually the depth data in addition to the 3D body 
skeleton of the person performing the action. 
However, in the context of hand gesture recognition, 
there are no publicly released dataset of dynamic 
hand gestures providing sequences of labelled hand 
gestures with the depth and hand skeleton. We present 
below a Dynamic Hand Gesture 14-28 (DHG) 
dataset, which provides sequences of hand skeleton in 
addition to the depth image. Such a dataset will 
facilitate the analysis of hand gestures and open new 
scientific axes to considerl . 

A Overview and protocol The DHG-14/28 dataset 
contains 14 gestures performed in two ways: using 
one finger and the whole hand.Each gesture is 
performed 5 times by 20 participants in 2 ways, 
resulting in 2800 sequences. Sequences are labelled 
following their gesture, the number of fingers used, 
the performer and the trial. Each frame contains a 
depth image, the coordinates of 22 joints both in the 
2D depth image space and in the 3D world space 
forming a full hand skeleton. The Intel Real Sense 
short range depth camera is used to collect our 
dataset. The depth images and hand skeletons were 
captured at 30 frames per second, with a 640x480 
resolution of the depth image. The length of sample 
gestures ranges goes from 20 to 50 frames. 


My Intel ii 



Figure 1. Shows sentence in English My father is in 
higher post and equivalent text in Kannada form is 
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3.2 Feature extraction from 3D skeleton 


In order to represent a hand gesture entirely, we 
propose to mainly capture the hand shape variation 
based on skeleton joints, but also the movement and 
the rotation of the hand in space are also computed. 
The temporal nature of gestures is encoded using a 
temporal pyramid and the classification process is 
performed by a linear Support Vector Machines 
(SVM) classifier. Figure 2 shows a general overview 
of the proposed approach. 




/* 


Figure 2.Shows pre-processing steps of each RGB 
frames from continuous gesture for the sentence My 
father is in higher post. 
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4- CONCLUSION 

This work suggests the advantage of using 3D hand 
skeleton data to describe hand gestures, and points out 
a promising direction of performing gesture 
recognition tasks using skeleton-like information. We 
present an approach to recognize dynamic hand 
gesture as time series of 3D hand skeleton returned by 
the Intel RealSense depth camera. We take as input a 
several set of relevant joints inferred from 3D hand 
skeleton. We propose a compact representation using 
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Fisher Vector kernel and on multi-level encoding the 
temporal nature of gestures. 

As future work, skeleton-based features can be 
combined with the depth-based features to provide 
more informative description and produce algorithms 
with better recognition robustness. 
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